Predicting face movements from speech acoustics using spectral dynamics
نویسندگان
چکیده
This paper introduces a new dynamical model which enhances the relationship between face movements and speech acoustics. Based on the autocorrelation of the acoustics and of the face movements, a causal and a non-causal filter are proposed to approximate dynamical features in the speech signals. The database consisted of sentences recorded acoustically, and by using a Qualisys system to capture face movements with 20 reflectors put on the face, simultaneously. Speech signals are represented by 16-order LSPs and log-energy. With the filtered dynamical features, the acoustic features account for more than 80% of the variance of face movements.
منابع مشابه
Classification of emotional speech using spectral pattern features
Speech Emotion Recognition (SER) is a new and challenging research area with a wide range of applications in man-machine interactions. The aim of a SER system is to recognize human emotion by analyzing the acoustics of speech sound. In this study, we propose Spectral Pattern features (SPs) and Harmonic Energy features (HEs) for emotion recognition. These features extracted from the spectrogram ...
متن کاملThe Dynamics of Audiovisual Behavior in Speech
While it is well-known that faces provide linguistically relevant information during communication, most efforts to identify the visual correlates of the acoustic signal have focused on the shape, position and luminance of the oral aperture. In this work, we extend the analysis to full facial motion under the assumption that the process of producing speech acoustics generates linguistically sal...
متن کاملCodebook Based Face Point Trajectory Synthesis Algo - rithm Using Speech
This paper presents a novel algorithm which generates three-dimensional face point trajectories for a given speech le with or without its text. The proposed algorithm rst employs an oo-line training phase. In this phase, recorded face point trajectories along with their speech data and phonetic labels are used to generate phonetic codebooks. These codebooks consist of both acoustic and visual f...
متن کاملEstimation of speech acoustics from visual speech features: A comparison of linear and non-linear models
This paper examines the degree of correlation between lip and jaw con guration and speech acoustics. The lip and jaw positions are characterised by a system of measurements taken from video images of the speaker's face and pro le, and the acoustics are represented using line spectral pair parameters and a measure of RMS energy. A correlation is found between the measured acoustic parameters and...
متن کاملOn the Relationship between Face Movements, Tongue Movements, and Speech Acoustics
This study examines relationships between external face movements, tongue movements, and speech acoustics for consonantvowel (CV) syllables and sentences spoken by two male and two female talkers with different visual intelligibility ratings. The questions addressed are how relationships among measures vary by syllable, whether talkers who are more intelligible produce greater optical evidence ...
متن کامل